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Mechanics Compared with Factor-Analytic Results’ 
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Michigan State University 


HE particular method of pattern 
developed in this paper is 
called agreement analysis. It can be in- 
troduced by contrasting it with factor 
analysis. These two methods differ in 
terms of the assumptions they imply con- 
cerning the interrelationships in subject 
matter they are designed to analyze. 
Agreement analysis is based on a gen- 
eralized theory of types. The types are 
assumed to reflect themselves in many 
kinds of interrelationships among char- 
acteristics—not just linear ones. Conse- 
quently, agreement analysis is a tool for 
determining all of the various ways in 
which characteristics are presumed to 
concatenate to yield types. Having this 
general ability to isolate types in terms of 
all kinds of interrelationships, it is, of 
course, capable of isolating them in terms 
of any restricted kind of interrelation- 
ships among characteristics. 

Factor analysis, on the other hand, 
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assumes that characteristics concatenate 
in terms of linear interrelationships only. 
It is a tool for analyzing the linear inter- 
relationships among characteristics and 
for expressing these in terms of factors. 
Types which may be discovered are 
limited either to standings on single fac- 
tors or configurations of standings on 
several factors. 


A more detailed contrast between factor analy- 
sis and agreement analysis will help give addi- 
tional meaning to the latter approach. The 
most common method of factor analysis (R- 
technique) has as its purpose to take the inter- 
correlations between a large number of indexes 
of personality and analyze these to yield a few 
fundamental personality traits in terms of which 
the indexes can be classified. 

An important consideration in this factor 
analytic procedure is that the intercorrelations 
are taken across all subjects of a sample; no 
allowance is given to the possibility that variables 
might be interrelated differently in various 
categories of subjects of a single sample. If 
variables are interrelated differently among 
categories of subjects and if the number of sub- 
jects representing each category varies con- 
siderably, then intercorrelation across all sub- 
jects as generally taken in factor analysis would 
represent a hodgepodge. 

Agreement analysis assumes types and is a 
method designed to assist in their isolation; it at- 
tempts to isolate them by classifying together 
individuals who give identical responses to large 
numbers of items of a test. 

Substantiation of a need for a method which 
contrasts with factor analysis is simple. Factor 
analysis does not prove the existence of factors; 
it merely clarifies the characteristics which they 
have if we postulate them, 

A limiting characteristic of factor analysis is 
the fact that it does not include a statistic for 
rejecting it as not fitting data particularly well. 
One way to help correct for this defect is to 
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develop methods based on the assumption of 
constructs other than factors, and compare re- 
sults obtained with the two methods. 

Agreement analysis postulates one kind of 
construct (types) which contrasts with factors. 
Applications of agreement analysis will help us 
to learn some of the characteristics which its 
types have. Comparisons of results from factor 
and agreement analyses will assist us in deter- 
mining which alternative postulate is more help- 
ful for various purposes. 

This paper includes four main sec- 
tions. The first sections develops the 
theory and method of agreement analy- 
sis and illustrates the method by applying 
it to descriptions of “best’’ and “poorest” 
mechanics. The second section compares 
the results of the agreement analysis with 
those of a factor analysis of common 
data, and interprets the types in the 
light of both the factor and agreement 
analyses. The third section critiques the 
method of agreement analysis. The 
fourth section summarizes the entire 


paper. 


I. CLASSIFICATION OF DESCRIPTIONS BY 
AGREEMENT ANALYSIS 


As mentioned above, an investigation 
by agreement analysis is based on a 
theory of types. In the particular theory 
of the present investigation, the concept 
of type is treated as an intrinsic con- 
struct, Each type is assumed to reflect 
itself in a pattern of characteristics, Con- 
sequently, individuals can be classified 
into typological categories according to 
the patterns of characteristics which they 
exhibit. Each category into which indi- 
viduals are classified is said to represent 
a type. The individuals of any one type 
are presumed to have certain character- 
istics in common. These characteristics 
are called a pattern. The individuals of 
any other type are also presumed to 
exhibit a pattern of characteristics, and 
their pattern is unique to their type. In 
brief, the individuals of each type ex- 


hibit a pattern .of common characteristics 
which is peculiar to them, For example, 
Type X might include individuals who 
exhibit a pattern of dependability, care- 
fulness, cooperativeness, and industrious- 
ness; Type Y might include individuals 
who exhibit the pattern of carefreeness, 
friendliness, joviality, and slipshodness. 


The Linnaean approach (1, pp. 490-492) to 
the classification of living organisms in the field 
of biology results in a chart which shows (a) 
all types of living organisms and (b) the pat- 
terns of characteristics peculiar to each type. At 
the lowest level of- classification of living things, 
each organism is classified into a species, Con- 
sequently, each species includes a number of 
organisms. The organisms of each species ex- 
hibit a number of common characteristics which 
are peculiar to them. 

At the next level of classification, each species 
is grouped with other species to form genera. 
Consequently, each genus contains several 
species. The species of each genus have certain 
characteristics in common which are peculiar to 
them. In an analogous fashion, genera are 
grouped to form families, families to form 
orders, and so on through classes, phyla, and 
kingdoms. 

A simplified and incomplete Linnaean chart 
is shown in Fig. 1 for illustrative purposes, At 
the bottom level of the chart, we have several 
individual organisms, then at the next level 
we have several species, then several genera, etc., 
up to a class. There is, of course, a pattern of 
characteristics for each species, genus, family, 
etc. 


Analogously to the simplified Lin- 
naean Chart, we propose to build a 
classification of certain descriptions of 
mechanics. 

Since so little is known about the types 
into which descriptions of mechanics may 
ultimately be classified, we propose for 
the present to use the Linnaean terms 
of species, genera, families, etc., for 
categorizing them. 

We have six purposes in building this 
classification; three of them are method- 
ological, and the other three are sub- 
stantive. The methodological purposes 
are (a) to illustrate a quantitative method 
for developing the classification, (b) to 
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Fic. 1. A simplified and incomplete Linnaean chart. 


indicate some of the potentialities of the 
method in the field of personnel research, 
and (c) to compare results from this pat- 
tern analytic approach with a factor 
analysis of common data. 

The substantive purposes are to make 
a preliminary investigation of the follow- 
ing hypotheses: (a) that the descriptions 
of mechanics selected as “best” and 
“poorest” can be classified into types; 
(b) that the classifications are relatively 
dependable in the sense that two separate 
classifications of subjects on the basis of 
two sets of test items will yield similar 
results; and (c) that the classifications 
are valid in the sense that they are re- 
lated to an external criterion. 

The above purposes are, of course, related to 
a theoretical, psychological position which 
should be specified unless it otherwise be mis- 
understood. We are not attempting, initially, to 
isolate the types which may in some way be said 
to be fundamental for the generalized under- 
standing of human behavior. For the time being, 
at least, it is assumed that different types may 
be significant for different purposes. For ex- 
ample, types which derive from an analysis of 
symptoms of diseases may be significant for 
differentiating between mentally and physically 
ill patients, but other types which derive from 
an analysis of characteristics of “best” and 
“poorest” mechanics may be required in differ- 
entiating subjects in relation to their qualifica- 
tions as mechanics, We are attempting to isolate 
types which are significantly related to a cri- 


terion, but we are not arguing that these single 
criterion types are the most fundamental ones in 
terms of which to understand behavior in 
general. On the contrary, we are inclined to the 
point of view that behavior can be categorized 
in many different ways and that each way will 
give somewhat different types. 

Even though we are interested initially in 
types pertinent to single criteria, we are, never- 
theless, also oriented ultimately toward the more 
fundamental scientific problem of deriving a 
parsimonious, systematic structure for the in- 
clusive representation of behavioral organization. 
Types which would fulfill this requirement 
would be minimal in number, simple in descrip- 
tion, and of maximal value in _ predicting 
behavior. 

The single-criterion studies can assist in the 
solution of the more fundamental problem. The 
types which derive from the single-criterion 
studies can be used as the basis for the develop- 
ment of those which are more generally mean- 
ingful, and therefore more basic, scientifically. 
Each single-criterion study will have selected a 
set of items, significant for the criterion under 
consideration. All of the items taken jointly 
from several single-criterion studies and analyzed 
simultaneously in such a manner as to isolate 
all the types represented in the data should 
produce those more comprehensive types which 
are pertinent to all the several single criteria. 
Consequently, these types should be relatively 
fundamental in the sense that they would be 
generally applicable in the prediction of be- 
havior. 

Since there is probably no absolute limit to 
the number of single criteria which may be used, 
and since each additional one may add some- 
thing, it will probably always be possible to im- 
prove somewhat on the classification into funda- 
mental types, which derive from this approach. 
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However, the rate of improvement is probably 
asymptotic, or at least would become so at some 
point, so that eventually the amount of im- 
provement which could be added would be- 
come very small. 


The Method of Classification 


We are now prepared to outline and 
illustrate with data a method of agree- 
ment analysis which we have developed 
for the isolation of types. It will here 
be outlined as it is applied in a single- 
criterion study. 

Subject and tests. Agreement analysis 
was applied to data which had been 
gathered in an earlier factor-analytic 
study by McQuitty, Wrigley, and Gaier 
(3). As pointed out in this earlier paper, 
an inventory of 200 items was developed 
using the language which mechanics used 
in describing other mechanics whom they 
had selected as representative of either 
“best,” “poorest,” or “average” me- 


chanics. The inventory for describing 
mechanics was administered to 428 su- 


pervisors, who selected a “best,” a 
“poorest,” or an “average’”’ mechanic and 
described him by replying either “yes” 
or “no” to each of the 200 items of the 
inventory. In the present investigation, 
we used the results from supervisors who 
described either a “best” or a “poorest” 
mechanic; we did not use the results from 
those who described an “average” me- 
chanic. 

In the first classification, we analyzed 
results from the first 120 items. In the 
second classification, we analyzed results 
from the remaining 80 items. The 200 
items of the inventory were listed in 
random order. The initial classification 
was based on the first 120 items because 
the factor analysis mentioned above was 
in terms of these items, and we wished 
to compare results from the two different 
kinds of analyses. 


We shall describe the first classification in 


detail; the second one will be clear by analogy. 
For each supervisor in the first classification, we 
had 120 responses which described a mechanic 
selected by him as either a “best” or a “poorest” 
mechanic. Each response is a description of a 
mechanic, Each 120 responses by any one super- 
visor is a pattern of descriptions. Each pattern 
of descriptions pertains to a particular mechanic, 
either a “best” one or a “poorest” one. 

There were 112 patterns pertaining to “best” 
mechanics and g2 pertaining to “poorest” me- 
chanics, giving a total of 204 patterns. Using 
random numbers, we arbitrarily reduced the 112 
patterns for “best” mechanics to 108 to give a 
grand total of patterns for “best” and “poorest” 
mechanics, combined, of 200 (for convenience 
of machine computations), 

Again using random numbers, we divided the 
108 patterns for “best” mechanics into equal- 
sized experimental and cross-validational samples. 
We did likewise for the 92 patterns descriptive 
of “poorest” mechanics. The two experimental 
samples, one for “best” and the other for 
“poorest” mechanics, were used to build our 
classification of patterns into types. The cross- 
validational samples were used to determine how 
well the types held up on other samples of the 
same universe, 

As the patterns of descriptions are classified, 
successively, into more and more inclusive types, 
some of the components of the patterns are 
dropped. In other words, the initial patterns 
with which we start are reduced in their num- 
bers of descriptions as the classification proceeds 
into more and more inclusive types. In order to 
differentiate these several “levels” of classification, 
we shall speak of individual patterns when 
classifying the initial ones into species; then of 
species patterns when we are classifying into 
genera; and of genus patterns when we are 
classifying into families; and so on. 


Details of the method. The method of 
analysis here used is an abbreviated ver- 
sion of a complete method of pattern 
analysis, developed by the author and 
called agreement analysis (6, 7). The 
present version differs from the complete 
method in these respects: (a) it omits a 
calculation which corrects agreement 
scores for chance fluctuations; (b) it ap- 
plies a somewhat less exacting, but more 
rapid, technique for classifying patterns 
into the categories which they resemble 
most; and (c) it limits each category to 
not more than two patterns from the 
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next lower level of classification; it is 
therefore a binary version. 

This abbreviated version of agreement 
analysis is justified here because the pur- 
pose is not to isolate the most funda- 
mental types but rather to study pat- 
terns of responses in relation to an ex- 
ternal criterion. This version allows the 
investigator to isolate rather rapidly a 
great number of reasonably reliable pat- 
terns and to study them as predictors. 


Our abbreviated method of binary agreement 
analysis involves a novel treatment of the agree- 
ment score developed by Zubin (10). The agree- 
ment score is computed for pairs of individual 
patterns, The agreement score for any two indi- 
vidual patterns is the number of identical de- 
scriptions contained in the two patterns. Suppose 
that the two individual patterns A and B both 
contain “yes” for the first five items of the 
inventory, disagree on the sixth item (one 
pattern having a “yes” and the other pattern a 
“no”), and that both patterns contain “no” for 
the next four items. On these 10 items, the two 
patterns, A and B, would have an agreement 
score of nine, because their answers would agree 
on nine of the 10 items. 

Each individual pattern was paired with every 
other individual pattern, and an agreement score 
was determined for each pair of individual pat- 
terns. These agreement scores are illustrated in 
Table 1 which shows 10 of the individual pat- 
terns descriptive of “best” mechanics. The 
patterns are labeled A through J and are so 
designated in the top row and left hand column 
of the table. 

The number, 102, in the intersection of row 
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A and column B of the table is the agreement 
score of the pair of individual patterns A and 
B. This same score also occurs in the intersection 
of row B and Column A. Other entries in Table 
1 are to be interpreted in an analogous fashion. 
Entries in the diagonal cells are omitted, since 
they would report the agreement score of each 
individual pattern with itself and are not needed 
in the analysis. 

In the present analysis, Table 1 was expanded 
to include the agreement scores for all 54 in- 
dividual patterns descriptive of “best’” mechanics 
in the experimental sample, i.e., the sample on 
which the classification of the descriptions of 
“best” mechanics was developed. Table 1 is shown 
as an abbreviated illustration of this larger 
matrix of agreement scores. 

The largest agreement score in the 54 « 54 
matrix was selected. Of all the individual] pat- 
terns represented in the matrix, the two with 
the largest agreement score have more answers 
in common than any other two. These two pat- 
terns constitute the first species. They and all 
agreement scores for either of them are with- 
drawn from the matrix. The pair of patterns 
with the highest agreement score in the reduced 
matrix is next selected. This constitutes species 
number two. The remaining species are se- 
lected in an analogous fashion. 

In the case of a tie in agreement scores, such 
as between the individual patterns of pair K 
and L, on the one hand, with M and N, on the 
other, it is immaterial as to which species is with- 
drawn first. However, in the case of a tie such as 
between X and Y, on the one hand, with X and 
Z, on the other, it is material. In this latter kind 
of tie with one pattern common to two pairs, 
we have followed the practice of classifying X 
with Y rather than Z only if the next highest 
agreement score in which Y occurs is lower than 
the next highest in which Z occurs, By selecting 


TABLE 1 
ILLUSTRATIVE SAMPLE OF “‘BEst’’ MECHANICS’ AGREEMENT SCORES BETWEEN INDIVIDUAL PATTERNS 


A B Cc D E F G H I J 

A 102 go 93 93 99 93 gl 92 99 
B 102 04 93 93 97 95 93 904 99 
€ go 04 IOI 905 97 07 98 100 
D 03 93 IOI 100 100 98 92 99 98 
E 93 93 101 100 102 96 92 95 04 
F 99 97 95 100 102 92 go 97 96 
G 93 95 97 98 96 92 100 97 100 
H gI 93 97 Q2 92 go 100 103 100 
I 92 94 98 99 95. 97 97 103 105 
J 99 99 IOI 98 04 96 100 100 105 


NOTE.—The table is to be read as follows: The agreement score between individual patterns A 
and B is 102; between individual patterns A and C is 90; etc. For definition of “individual pattern” 
and ‘‘agreement score,”’ see text. 
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Y instead of Z, the agreement score which will 
be used to classify Z is larger than would be the 
one to classify Y if we had reversed our choice. 
If the second highest agreement scores are also 
tied, reference can be made to the highest ones 
which are not tied. These can then be used as 
a basis for a decision. 

We have now outlined how to classify all 54 
individual patterns into species, each species con- 
taining two members, and only two. In those 
studies where there is one individual pattern 
left over, it is classified into an additional species 
containing only one case. 

The classification of individual patterns into 
species started with a pattern of answers for 
each mechanic described. The pattern of answers 
is the endorsements on the particular inventory 
used to describe the mechanic. The classification 
into species brings together those pairs of in- 
dividual patterns which are highly similar, as 
determined by agreement scores. We can use 
this same approach for classifying species into 
genera, provided we can obtain a pattern of 
answers for each species. 

A species’ pattern of answers is those endorse- 
ments which are common to both individual 
patterns of the species. Suppose, for example, 
that both individual patterns X and Y of a 
species have “yes” for item 1 and “no” for item 
2. Their species pattern for the first two items 
would be “yes’-“no.” Suppose further that in- 
dividual pattern X has “yes” for item 3, while 
individual pattern Y has “no” for this item, and 
they both have “yes” for item 4. Their species 
pattern for the first four items would be “yes”- 
“no”-blank-“yes.”. The blank denotes the fact 
that the two individual patterns disagree on 
item g. This item is dropped from consideration 
in the further classification of this species. In 
other species where both individual patterns 
have the same answer to the item, it is retained. 

In the manner just described, a pattern of 
answers is determined for each species which 
contains two individual patterns. These are 
called species patterns. In those studies where 
the analysis started with an odd number of cases 
and ends in the last species containing but one 
individual pattern, the pattern of answers for 
this species is the same as those of the one in- 
dividual pattern which the species includes. 


Since we now have a pattern of an- 
swers for each species, we can use the 
same technique for classifying species 
patterns into genera as we originally 
used for classifying individual patterns 
into species. Further, we can use the 
same plan for determining the pattern 


of answers for each genus as we used 
for determining the pattern of answers 
for each species. The system is entirely 
repetitive at each successive level; we 
merely continue it for classifying species 
patterns into genera, genus patterns into 
families, etc., until the classification is 
completed. 

In an analogous fashion as outlined 
above, we classified the patterns descrip- 
tive of “poorest” mechanics. This classi- 
fication had to be discontinued at the 
class level, where there were only two 
classes, because the patterns for classes 
contained common responses to no item. 

Unique features of the method. The 
method of analysis contains a number of 
unique features. It is essential to ap- 
preciate these if the implications of the 
method are to be understood and if re- 
sults from it are to be properly inter- 
preted. These unique features can be 
introduced by showing how the method 
is designed to capitalize on the predictive 
possibilities of responses treated in com- 
binations. 

Meehl (g) has illustrated that it is 
theoretically possible for responses 
treated in pairs to have relationships to 
a criterion even though the items treated 
separately have zero relationship to a 
criterion. This potentiality of responses 
treated in pairs would appear to make 
it highly desirable to analyze responses 
in combinations of two. However, 100 
two-answer items taken two at a time 
would yield 4,950 pairs of items and 
19,800 response pairs for analysis. To 
analyze this many combinations would 
be an unreasonably laborious task. Be- 
sides, to analyze in combinations of two 
is not an entirely satisfactory solution. 
For, if pairs have potentialities over and 
above separate items, then by the same 
logic triads have potentialities over and 
above pairs, and likewise any order of a 
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combination has potentialities over and 
above any one of lower order. We are 
forced then to the conclusion that it is 
essential to treat items of a test in rela- 
tively large combinations if we are to 
investigate thoroughly the predictive po- 
tentialities of the test. 

To analyze all items of a test in com- 
bination is not nearly as laborious as it 
would at first seem. Even though there 
are many combinations of responses theo- 
retically possible, only a very few of them 
occur empirically, and those which do 
not occur need not concern us. 

In those patterns which do occur, some 
of the responses are relatively undepend- 
able and may be disregarded. A problem 
is to determine which responses are rela- 
tively undependable. In our approach to 
this problem, we assume that a response 
may be completely dependable for some 
subjects and completely undependable 
for other subjects. 

This assignment of differential, all or 
none, dependability of responses in rela- 
tion to the subjects who give them is 
novel in test analysis. The usual ap- 
proach is to assign a single index of 
reliability for each item as computed 
from the average behavior with respect 
to it, even though some subjects may al- 
ways give the same response to an item 
while others continually change in their 
responses. 

Our approach, on the other hand, as- 
signs a perfect reliability to an item for 
those subjects whose responses to it are 
consistent with their responses to other 
items. For the same item, our approach 
assigns a zero reliability for those sub- 
jects whose responses to this item are 
inconsistent with their responses to other 
items. 

In an effort to determine which responses are 


consistent with other responses, we classify sub- 
jects together as already explained in our method 


of agreement analysis. In this approach, some 
items are irrelevant (undependable) in the sense 
that they do not classify a pattern in the same 
fashion as the majority of the responses in a 
pattern. As an illustration, suppose that a test 
of 100 items is administered to 100 subjects, 
and that subjects A and B agree in the answers 
which they give to 99 of the 100 items. Suppose 
further that the next highest agreement score 
that either of these subjects has with any other 
subject is 80. We would conclude that A and B 
should be classified together in terms of their 
patterns of responses to the items of the test, 
and that their responses to the one item on 
which they disagree is irrelevant to their classi- 
fication. Their classification would be achieved in 
terms of their predominant patterns of responses, 
where these patterns are defined as the ones 
which include the greatest possible number of 
responses and still yield a classification. 

Consequently, any response which does not 
conform to the predominant pattern of responses 
is presumed to be irrelevant. It is assumed to 
vary independently of the responses in the pre- 
dominant pattern. In computing a correction for 
agreement scores in the complete method of 
analysis (6, 7), irrelevant responses are as- 
sumed to vary by chance with respect to the 
predominant patterns. 

A response which is irrelevant in the classifica- 
tion of one subject may, however, be relevant 
in the classification of another subject. Suppose 
that the one item on which subjects A and B 
of the above example disagree is item 2; their 
responses to this item are irrelevant in classifying 
them together. Suppose further that C and D 
are classified together because they agree on 99 
(of 100 items) including item 2 but excluding 
item g. In this case item 3 rather than item 2 
would be designated as irrelevant; item 2 would 
be relevant for subjects C and D but not for A 
and B. 

As the classification by agreement analysis 
proceeds from the species levels to higher ones, 
the criterion for judging responses to be relevant 
usually becomes more stringent. In order for a 
response to an item to be relevant in the first 
classification, a response to it has only to be 
given by both of two subjects who classify to- 
gether. As the classification proceeds, more and 
more individuals are classified into fewer but 
larger categories and all of the subjects of any 
one category have to agree on a response in 
order for it to be relevant to their classification. 

Since irrelevant responses are assumed to be 
independent of the predominant patterns which 
determine the classification, they are presumed 
to occur by chance with respect to the pre- 
dominant ones. Consequently, irrelevant re- 
sponses can be expected to yield both agree- 
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ments and disagreements between predominant 
patterns. We can spot those which yield dis- 
agreements as shown above. From these, we can 
estimate the number of irrelevant responses 
which yield agreements in any classification. For 
example, if subjects A and B who agreed on 99 
out of 100 items of a test were responding to 
items which have only two possible answers, we 
would expect that their agreement on irrelevant 
responses would equal their disagreement on 
irrelevant responses. Since they disagreed on one 
irrelevant item, we would estimate that they 
agreed by chance on one irrelevant item and 
we would further estimate their true agreement 
score to be 98 rather than gg. This example 
illustrates the logic on which our corrected 
agreement score is based for the complete ver- 
sion of agreement analysis (6, 7). 

It is helpful to emphasize the assumption on 
which the correction is based in order that 
agreement analysis can be properly understood. 
The correction does not assume that each answer 
alternative is equally apt to occur, as many col- 
leagues have originally thought when they first 
examined the method. It assumes instead that 
subjects are equally apt to give any one of the 
possible answer alternatives to those particular 
‘items which are irrelevant for their classification 
into predominant patterns. 


Results from the Classification 


A portion of the results from classify- 
ing descriptions of “best” mechanics is 
shown in Table 2; space does not permit 
showing all of them. In the table, the 
lowest level of classification represents 
the individual patterns; the next level 
represents species; and then a genus is 
shown. This table corresponds to only 
a small portion of the chart shown as 
our model in Fig. 1. It corresponds to 
any section which includes one genus, 
two species, and four individual pat- 
terns. 


Table 2 shows certain detailed information 
not shown in Fig. 1. The capital letters of Table 
2 are codes for patterns descriptive of “besi” 
mechanics. Thus the entry on the left side of 
the table at the species level means that this 
species includes individual patterns K and L. 
Under this species is a twice underlined number, 
106; this is the number of items on which in- 
dividual patterns K and L agree. It is their agree- 
ment score, out of a total of 120 times, At the 
genus level, we find the codes K-L and H-I. 


AN ILLUSTRATIVE CIASSIFICATION OF DESCRIPTIONS OF "BEST" MECHANICS 


Gemes K-L-H-I 


Agreement score: 86 


K-L 
106 
= 


Species: 
Agreement score? 


"No" Ans.t 8 32 33 
Z 61, 70, 75, 89, 103 12 


"Yes" Anse: 2, 5, 26, 31, 
56, 77, 80,101, 16 


Species: H-I 

Agreement score: be 
"No" Ans.? 15, 52, 

Ty $0, 86, 


"Yes" Ans, 9, LO, Ug, 
59, 8, 87, 96, 12 


Individual Pattern: K 


Individual Pattern: L 
"Yes" 


BES 


Individual Pattern: H 
"Yes" Ans.2 


Individual Pattern: I 
"Yes" Ans.? 


22, 
10, 


Capital letter = codes for patterns. 
Double underlined mumbers © agreement scores. 


Other numbers © codes for items. The code mumbers listed immediately below each of the two species, for example, represent 
the items for which both individual patternsof the species have the same answer, but for which at least one individual 


pattern of the other species has the 
have the opposite answer, the code 

= at the next higher classification as irrelevant. 
tems. 


te answer, 


In cases where both individual patterns of the other species 
S are underlined, Only those items are listed at each level which are 


The two individual pattern of species K-L disagree on 1) 


These items are listed under both individual patterns K and L, but with opposite answers in the two places; 


the individual patterns of the species agree on the other 106 items of the inventory. 


~~ 
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These letters mean that this genus includes 
species patterns K-L and H-I, which were de- 
rived from individual patterns K, L, H, and I. 
They agree on 86 items as shown by the twice 
underlined number, 86, listed under the code 
letters for the genus. Other code letters and 
other twice underlined numbers of Table 2 are 
interpreted in an analogous fashion. 

The species are arranged from left to right 
according to the size of agreement scores, with 
the largest ones on the left. In other words, as 
we move from left to right across charts such 
as the one of Table 2, the classifications generally 
become less dependable, simply because we thus 
generally obtain successively smaller agreement 
scores, and they are successively less dependable. 

In a complete chart of results (which could 
be prepared from the analysis), all of the items 
and responses to them would be listed in such 
a manner as to show both the exact answers con- 
tained in any pattern, and which items differ- 
entiate various individual patterns, species, 
genera, etc.,, one from another and to what 
extent, These points can be illustrated by refer- 
ence to the incomplete chart of Table 2. Con- 
sider the items listed immediately below either 
of two species which classify in a genus, The 
items below either of these two species are the 
ones for which each individual pattern of the 
species has the same answer, and for which at 
least one individual pattern of the other species 
has the opposite answer; sometimes both in- 
dividual patterns of the other species have the 
opposite answer. In this latter case the code 
numbers for the items concerned are underlined. 
The items are classified according to whether the 
agreement is in terms of a “yes” answer or a 
“no” answer. For example, both individual pat- 
terns of species K-L have a “yes” on the list of 
items beginning with 2, 5, 26, etc; and both 
individual patterns of species H-I have a “no” 
on the list of items beginning with 5, 15, 52,. etc. 
Items 5, 80 and 116 are common to these two 
groups, as indicated by the single underlines. For 
these two species, the latter three items are per- 
fectly positively correlated with each other and 
perfectly negatively correlated with the other 
underlined items of the species, i.e., items for 
which species K-L have “no” answers and species 
H-I have “yes” answers. 

A complete chart would show the entire de- 
scription contained in each individual pattern, 
each species pattern, each genus pattern, etc. 
This can be illustrated by reference to the in- 
complete chart of Table 2. Suppose that we want 
to know the composition of individual pattern L. 
The answers to the items listed immediately 
below it give only a few of the answers which it 
includes; some of the others are the answers to 
the items listed for the species containing the 
letter L, namely species K-L, Stil others are 


the answers of the items for genus K-L-H-I. 
These latter answers are not shown in the incom- 
plete chart of Table 2 because of lack of space. 
Analogously, as would be shown on a complete 
chart, all of the additional answers of individual 
pattern L, would be listed for the family, order, 
class, and phyla which contain the letter L. 
The classification was completed at the phylum 
level because all classes came together in a single 
phylum. The description of any other initial 
pattern or any species, genus, family, etc., can 
be read in an analogous fashion from a complete 
chart. We see then that there is a very unusual 
and important feature of a complete chart. Even 
though it is the terminal table of a method 
which has analyzed data in order to facilitate 
interpretation of it, no piece of information is 
lost; the response of each subject to each item 
is shown. 

The above described feature of retaining 
every piece of information in the summary table 
(derived from the analysis) is not realized with- 
out creating a problem. One purpose of scientific 
inquiry is to abstract that material which leads 
to generalizations. If all information is retained, 
no abstracting has occurred. However, abstract- 
ing can occur at various levels. In agreement 
analysis, each successive level of classification 
represents an increase in the degree of abstrac- 
tion. Consequently, the question arises as to 
which level yields the most meaningful results. 
From a purely practical point of view, we can 
argue in favor of that level which yields the 
closest relationship with an external criterion. 
However, this principle would appear to be a 
rather tenuous one on which to base funda- 
mental, scientific decisions. A better scientific 
basis can be suggested if the problem is con- 
sidered in the light of the theory out of which 
agreement analysis developed. This theory postu- 
lated real types. If real types exist, they may be 
either concise and discrete or more amorphous 
in nature. If they are discrete, then they should 
reveal themselves most cogently at some one level 
of classification. This would probably be the 
level of classification at which the agreement 
scores have the highest statistical significance. 
Consequently, statistical theory of significance 
could be applied in an effort to determine the 
level of classification in which real types are por- 
trayed. Some statistics designed to evaluate the 
significance of agreement scores have been re- 
ported elsewhere (4). 

On the other hand, suppose that types exist 
only in a relatively amorphous sense; those at 
any one level are ill-defined and fade into those 
at the next higher level. Under these conditions, 
there might be no very fundamental basis for 
stating that the types at any one level are more 
basic than those at any other level. The level of 
primary concern would then have to depend on 
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some utilitarian consideration, such as the degree 
of relation to an external criterion. 

There is another unusual feature of agree- 
ment analysis. Since the analysis is based on com- 
binations of answers, and since no combination 
whatsoever is excluded by the method, the 
method automatically handles whatever kinds of 
interrelationships (linear, curvilinear, and dis- 
junctive) are involved in the data, without in 
any way restricting them. The method also allows 
any specific answer to be given various meanings, 
or interpretations, depending on the combina- 
tions or patterns of other answers with which 
it occurs. 


Reliability of the method. In order to 
investigate the reliability of the method 
of classifying subjects into categories, the 
100 subjects of the experimental sample 
who were originally classified on the 
basis of the first 120 items of the in- 
ventory were again classified on the basis 
of the last 80 items of the inventory. In 
the original analysis, the 54 descriptions 
of “best” mechanics and the 46 descrip- 
tions of “poorest” ones were classified 
separately. This same approach was fol- 
lowed here in classifying the subjects on 
the basis of 80 items, thus resulting in 
two independent studies of reliability, 
one on descriptions of “best” mechanics 
and the other on descriptions of “poor- 
est” mechanics. 


Each analysis was continued until every in- 
dividual was classified into one of two large cate- 
gories at the top level of classification, However, 
we did not wish to assess reliability at this highest 
level exclusively, because as already pointed out 
we planned tentatively to study later the validity 
at all levels of classification and we wished to 
know whether or not the classification was suffi- 
ciently dependable to justify validity studies. For 
this latter purpose, it did not appear necessary 
to study reliability at all levels, if we could gain 
an overview from selected portions. Consequently, 
for the purpose of comparing the classifications 
on the basis of 80 items with the earlier ones on 
the basis of 120 items, we selected categories 
which were intermediate in the analysis. The 
categories were selected so that every subject was 
classified once and only once for the 80-item 
study, and likewise for the 120-item study. In 
the case of “best” mechanics, we selected for 
both studies the first class (two orders) of 32 sub- 


jects resulting from the analyses and the two 
remaining orders of 16 and 6 subjects, In the 
case of “poorest” mechanics, we selected the first 
order (two families) of 16 subjects and the four 
remaining families of 8, 8, 6, and 8 subjects. The 
categories of “best” mechanics were ordered in 
each analysis according to the number of test 
items on which the patterns of each category 
agreed. In the 120-item analysis of “best” me- 
chanics, the 32-subject class agreed on 43 items, 
the 16-subject order on 31 items and the 6-subject 
order on six items. These categories were desig- 
nated 1, 2, and g respectively. In the 80-item 
analysis, the 32-subject class agreed on 36 items, 
the 16-subject order on 24 items and the 6-subject 
order on 13 items. These categories were desig- 
nated a, b, and c respectively. 

To the extent that categories a, b and c have 
common members with categories 1, 2 and 3, re- 
spectively, the method of analysis is reliable 
across items. The correspondence between these 
two sets of categories is shown in Table 3. Thirty- 


TABLE 3 


ASSOCIATION BETWEEN Two AGREEMENT 
ANALYSES OF COMMON SUBJECTS 
(“Best’’ MECHANICS) 


Categories | Categories From First 120 Items 
From Last 
80 Items 


3 2 I 


a 
b 


| 

| I 23 
I 8 

4 


four of the 54 cases, 63 per cent, have identical 
classification in the two analyses. Combining 
categories g with 2 and 6b with c to obtain a 
22 table (with larger expected frequencies), 
we computed the chi square and found it to be 
5.12, significant at about 0.02. (Chi squares of 
5-41 and 3.84 are required for p—0.02 and 
p = 0.05, respectively.) 

The analogous results for the 5 categories of 
the “poorest” mechanics are shown in Table 4. 


TABLE 4 


ASSOCIATION BETWEEN Two AGREEMENT 
ANALYSES OF COMMON SUBJECTS 
(‘‘PoorEst” MECHANICS) 


Categories | Categories From First 120 Items 
From Last 


80 Items 8 7 6 5 


d 8 

e I 4 3 

f | 5 5 2 4 

4 I 5 

h 2 I 2 2 I 


A PATTERN 


Twenty-four of the 46 cases, 52 per cent, have 
identical classifications in the two analyses of 
“poorest” mechanics. Combining categories 8, 7, 
and 6; 5 with 4; f, g, and h; and d with e to 
obtain a 2x2 table (with larger expected fre- 
quencies) we computed the chi square and found 


it to be 16.81, significant at better than 0.001. 


(The chi square for p= 0.001 is 10.83.) These 
two sets of results show that binary agreement 
analysis gives reasonably dependable results even 
when applied to relatively homogeneous groups 
such as “best” or “poorest” mechanics, 


Validity of the method, The two classi- 
fication schemata from the original analy- 
sis based on 120 items (one for patterns 
descriptive of 54 “best” and the other 
for patterns descriptive of 46 “poorest” 
mechanics) were investigated as tools for 
differentiating between cross-validational 
samples. In order to simplify a discussion 
of the method used, suppose we have an 
individual pattern X, which is not one 
of the experimental patterns, and sup- 
pose its categorization as between “best” 
and “poorest” is kept a secret until we 
classify it in terms of the two schemata. 
We then check to see if the classification 
is correct. In order to classify pattern X 
in terms of the two schemata, we first 
obtain an agreement score for it with 
each of the individual patterns of the 
experimental samples. On the basis of 
these agreement scores, we classify indi- 
vidual pattern X in one of three ways, 
namely, (a) as a pattern descriptive of a 
“best” mechanic, if its highest agreement 
score is with an experimental “best” in- 
dividual pattern, (b) as a pattern de- 
scriptive of a “poorest” mechanic, if its 
highest score is with an experimental 
“poorest” individual pattern, or (c) as 
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indeterminate, if there is a tie and indi- 
vidual pattern X has its highest agree- 
ment score with both a “best” and a 
“poorest” experimental, individual pat- 
tern. The classification schemata are 
valid for individual pattern X if they 
classify it correctly. 

However, in the above procedure, we 
have not yet exhausted all of the po- 
tentialities of the classification schemata. 
In fact, we have hardly used the schemata 
at all. We have scored individual pattern 
X in terms of individual patterns only. 
We can also score it in terms of the pat- 
terns of species, genera, families, etc., for 
“best” and “poorest” experimental data 
and classify it as either “best” or “poor- 
est” at each of these levels. A problem 
is to discover which level gives results 
in closest agreement with the criterion 
classification. This problem is investi- 
gated by classifying many cross-valida- 
tional, individual patterns as “‘best’’ or 
“poorest” at each level and analyzing the 
results to determine which level gives 
the highest percentage of correct classifi- 
cations. We did this for the individual 
patterns of our cross-validational sam- 
ples. 

The results are shown in Table 5, 
which reports the percentages of correct 
classifications for cross-validational, in- 
dividual patterns descriptive of “best” 
and “poorest” mechanics, shown sepa- 
rately, and then also combined, with 
each category weighted equally. For ex- 
ample, at the species level, 100 per cent 
of the individual patterns for “best” 


TABLE 5 
PERCENTAGES OF CoRRECT CLASSIFICATION ON THE CROSS-VALIDATION SAMPLE (120 ITEMS) 


Classification 


‘Individuals Species 


Genera 


Families Orders Classes Phyla 


“Best” 
“Poorest” 
Combined 


100 
13 
57 


| 
. 96 100 100 100 100 100 
89 83 76 7° 7° 33 
93 92 88 85 85 67 
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mechanics and 83 per cent of those for 
“poorest” mechanics were correctly classi- 
fied. A mean of these two, namely 92, 
is the average percentage of the combined 
groups correctly classified. The maximum 
percentage of correct classifications for 
the individual patterns descriptive of 
“poorest” mechanics is 89 and occurs at 
the individual level. For those descriptive 
of “best” mechanics, it is 100, for all 
levels above that of the individual pat- 
terns. For “best” and “poorest” com- 
bined, and equally weighted as groups, 
the maximum percentage of correct 
classifications is 93 and occurs at the 
individual pattern level. 

Multiple types of “best” and “poorest” 
descriptions. There is one very interesting 
characteristic of the cross-validational 
percentages. The largest percentage for 
the individual patterns descriptive of 
“poorest” mechanics is at the individual- 
pattern level, with 89 per cent correctly 
classified; the percentages then show a de- 


crease down to only 33 per cent correctly 
classified at the class level. In other words, 
as descriptions of both “best” and “poor- 


est” mechanics are classified separately 
into more inclusive categories, the items 
in terms of which the “best” and “poor- 
est” descriptions differ drop out of the 
analysis. These items which differentiate 
as between “best” and “poorest” descrip- 
tions also differentiate within both the 
“best” and “poorest” groups and conse- 
quently yield many types of both “best” 
and “poorest” descriptions. On the 
other hand, the items in terms of which 
many “best” descriptions agree have 
identical or similar responses for descrip- 
tions of “poorest” mechanics; “poorest” 
descriptions do not differ from “best” 
ones primarily in those characteristics 
which are common to “best” descriptions, 
but rather in those which are relatively 


heterogenous for even “best” descrip- 
tions. Our analysis revealed no single 
ideal type of “best” mechanic which dif- 
fers from an all-inclusive prototype of 
“poorest” mechanic; multi-types seem to 
be required if maximum differentiation 
is to be obtained between categories of 
“best” and ‘“‘poorest” mechanics. 
Hypotheses analogous to the ones just outlined 
have derived from a similar study by the writer 
which compared mental hospital patients with 
community subjects (5). Mental hospital patients 
do not seem to differ from community subjects 
on those characteristics which are common to 
almost all community subjects. Instead, they ap- 
pear to differ more on those characteristics which 
are peculiar to species and genera of community 
subjects. Multi-types of both patients and com- 
munity subjects appear to be required in order 
to achieve maximum differentiation. These hy- 
potheses might prove helpful in areas which 
have frustrated quantitative efforts to differ- 
entiate categories of individuals satisfactorily. 
Improvement through item selection. 
The fact that the percentage of correct 
classifications decreases as we move from 
species to higher level categories suggests 
the hypothesis that items which are rele- 
vant to the greatest number of species 
are the most effective in obtaining cor- 
rect classifications. However, some items 
are relevant for both species and higher- 
level categories. In fact, in order to be 
relevant for a higher-level category, an 
item must be relevant for all species 
classified in it. But items which are rele- 
vant at both species and higher levels 
would fail to yield the species. In order 
to differentiate between species, we must 
have some items which are relevant at 
the species level only. In attempting 
to select a reduced number of items 
which would be equally as effective as 
the original 120 items, we selected 26 
items with the following characteristics: 
(a) each item selected must be relevant 
for at least 82 per cent of the species 
isolated; (b) some of the items selected 
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must be relevant primarily at the species 
level only; (c) others must be relevant at 
higher levels as well. As a consequence, 
the items selected varied in relevancy at 
the order level from those which were 
relevant for only 16 per cent of the 
orders up to those which were relevant 
for 71 per cent of the orders. 

Using these 26 items, in lieu of the 
original 120 items, we repeated both 
the experimental and cross-validational 
studies. As a control group of items 
against which to compare the above se- 
lected sample, we chose 26 items at 
random from the 120 original ones and 
also repeated the studies on them. 

The percentages of correct classifica- 
tions derived from 26 especially selected 
items on the cross-validational samples 
is shown in Table 6. It is analogous to 


TABLE 6 


PERCENTAGES OF CorRECT CLASSIFICATIONS 
FOR CROSS-VALIDATION SAMPLES (26 
ITEMS OF KNOWN RELEVANCIES) 


Classifi- 


Indi- Spe- 
cation 


vidual cies 


“Best” 
“Poorest” 
Combined 


Table 5,: which was based upon 120 
items. For combined samples (“best” and 
“poorest”) of. the cross-validational sub- 
jects, the 26 items give higher percent- 
ages of correct classification than the 
120 items at the individual, species, 
genus, and family levels; at the order and 
class levels, the 120 items give the higher 
percentages. The highest percentage of 
correct classifications, namely g5, is ob- 
tained at the species level for the 26 
items. 

The percentage of correct classification 
derived from the 26 randomly selected 
items for the cross-validational subjects 
are shown in Table 7. This table is anal- 


TABLE 7 


PERCENTAGES OF CorRECT CLASSIFICATION, 
FOR CROSS-VALIDATION SAMPLES, 
26 RANDOM ITEMS 


Classifi- 


Orders Classes 


“Poorest” 
Combined] 


ogous to Table 5, based upon the 120- 
item study, and to Table 6, based upon 
the 26 especially selected items. 

The 26 especially selected items gave 
the two highest percentages of correct 
classification, 94 and g5, and tied with 
the 120 items for the third highest, 93. 
In addition to this tie, the 120-item test 
gave the fourth highest percentage, ge. 
The 26 random items gave the fifth high- 
est, go, and tied with the 120-item test 
on the sixth highest, 88. 


Since the purpose of selecting items was to 
obtain those most effective at the species level, 
we computed the significance of the difference in 
percentage of correct classifications at this level 
between the 26 especially selected items and the 
26 items chosen at random. Table 8 shows the 


TABLE 8 


ACCURACY OF CLASSIFYING SUBJECTS WITH 
RANDOM vs. SELECTED ITEMS 


26 Random 26 Items of Known Relevancies 


Items 


Incorrect Correct 


Correct ° 88 
Incorrect 5 7 


data for this computation, assuming a correlation 
between the percentages. However, this table 
does not meet the minimal requirements for a 
computation of this kind; one cell has a zero 
entry and the cell diagonally below it has only 
seven as an entry (2, p. 80). A test which fails to 
assume a correlation can, however, be applied to 
this data provided the result is interpreted with 
the understanding that it is an underestimate of 
the significance of the difference to the extent 
that the percentages are correlated, as they would 
normally be in a situation of this kind. A test 
of this kind (2, p. 76) was applied. It produced 


78 70 78 76 72 48 
89 88 89 88 86 74 
Gen- Fam- 
Orders Ciasses 
100 100 100 100 
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a critical ratio of 1.88, which is significant at the 
0.03 level in a single-tail interpretation, as is 
here appropriate. 

The empirical results reported above 
support the hypothesis that items which 
have high relevancies for species and vary 
in their relevancies for higher-level types 
give higher percentages of correct classi- 
fications than an equal number of ran- 
domly selected items. 


tions, there is sometimes a possibility of improv- 
ing on the results. This possibility derives from 
the fact that one application of agreement analy- 
sis to all of the items of a test isolates only the 
predominant types. The responses which do not 
fit these types are rejected at the various levels 
of classification. The pattern analysis could be 
repeated on the rejected responses. These would 
yield species not isolated from the analysis of the 
test as a whole. They might give correct classifi- 
cation missed by the other items, In the present 
study, we had relatively few incorrect classifica- 


tions and relatively few responses rejected at the 
species level. As a consequence, we did not try 
a reclassification on the basis of rejected items; 


In cases where an agreement analysis fails to 
give a satisfactory percentage of correct classifica- 


TABLE 9 
43 Irems DESCRIPTIVE OF THE First CLass (2 OrpERS) or MECHANICS (N= 32) 


Item No. Response Item 


He can get along with people pretty well. 

When he does a job, you know it will be done right. 

He answers questions as best he can. 

He knows how to use tools. 

His personal traits are O.K. 

You can take his word for what he says he’s done. 

He deserves a promotion. 

You don’t have to worry about telling him what to do all the time. 
He sees decision through to an end. 

He uses the right tools when he can get them. 

He is very glad to help somebody else. 

He makes sure he does a good job. 

He tries to find better ways of doing things. 

His work is nearly 100% correct. 

The pilots all seem to get along with him pretty well. 

What he doesn’t know he can find out for you one way or another. 
Everyone enjoys working with him. 

He knows his stuff. 

If you leave him to doa job, you can always be sure he will get the job done. 
What he knows, he never forgets. 

He usually remembers things which are explained to him. 

He can show you how to do the job right. 

His ambition will pay off. 

He is good at working on the plane. 

He gives good cooperation. 

He seems to take pride in his work. 

He is more often wrong than right in his decisions. 

He works in a sloppy way. 

He is afraid to make a decision one way or the other. 

Most guys with that much experience know a lot more than he does. 
He just stands around until you're done before offering to help. 
He doesn’t have any sense of responsibility. 

He’s just careless. 

He isn’t a very careful worker. 

He achieves his aim in the wrong way. 

He’s always getting messed up in money deals. 

He is kind of slipshod in his ways. 

When he does get over to work, he doesn’t do a thing. 

You have to explain every little step to him. 

No cooperation at all. 

Sometimes you can trust him and sometimes you can’t. 
Instead of doing it, he says, “I'll think about it.” 

He ‘‘goofs-off” all day long. 
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a reclassification on the basis of the present data 
would not represent a critical evaluation of the 
proposal, and it is too laborious to apply it to 
noncrucial data just because the latter are avail- 
able? 

In the results of the several analyses 
reported herein, it is helpful to mention 
that all of them agree in differentiating 
more “poorest” descriptions from “best” 
ones at the lower levels of classification, 
where there are relatively many cate- 
gories and each one classifies only a few 
individual patterns, rather than at the 
higher levels, where there are few cate- 
gories and each classifies a large number 
of individual patterns under it. 


II. A Jomnt CONSIDERATION OF AGREE- 
MENT AND Facror-ANALYTIC RESULTS 


In this section types are interpreted in 
terms of results from both agreement and 
factor analyses. Then, the interassocia- 
tions between factors and types are 
studied further in terms of items which 
are common and distinct for the types 
and factors. 


Interpretation of the Types 


Several of the primary advantages of 
agreement analysis are realized when we 
attempt to interpret the types. Each type 
is broad in the kinds of information fur- 
nished about the subjects who constitute 
the type, provided the test is wide in its 
coverage. This point is illustrated by list- 
ing the characteristics which were found 
in this study to belong to the first class 
(32 subjects) of “best” mechanics iso- 
lated. These characteristics are shown in 
Table g, with the items which were 
answered “Yes” listed first, and those 


*In addition, we have recently developed a 
method of multiple classification by agreement 
analysis whereby a subject can be classified in 
terms of all of the category patterns which are 
contained in his individual pattern. 


answered “No” following. 

The diversity of characteristics repre- 
sented by these descriptions is illustrated 
by considering several of the items at the 
beginning of the above list. The first one 
tells about human relationships, the next 
one about quality of job performance, 
the next about reactions to questions, 
then about use of tools, personal traits, 
truthfulness, etc., in the order named, 
with each one giving information in a 
different area of behavior. 

The diversity of information furnished 
about each type is further substantiated 
by a brief comparison of results from the 
agreement analyses with those from a 
square root, unrotated factor analysis of 
common data. 


The 120 items which were analyzed here by 
two separate agreement analyses, using first 54 
“Sest” mechanics and then 46 “poorest” ones, 
previously had been factor analyzed using 428 
subjects (3), including those used in the agree- 
ment analyses. Using the results of this factor 
analysis, we examined the 43 items listed above 
as descriptive of the first class of “best” me- 
chanics. These 43 items represent substantial 
loadings on 13 of the 23 factors reported in the 
factor analysis, Each of these 13 factors contains 
at least two items from the above 43 with loadings 
in its upper ten, and nine of the 13 factors con- 
tain four or more items (from the 43) with load- 
ings in the top ten per factor, The first factor has 
all of its 10 highest loaded items represented in 
the 43 descriptions of the first class of “best” 
mechanics. The next two factors have none 
represented; the fourth has eight in the top 
ten. The others are as follows, with the fac- 
tor number listed first, followed by the num- 
ber of items from the first class with loadings in 
the top ten:* 5-4, 6-5, 7-0, 8-5, 9-6, 10-0, 11-1, 
12-5, 13-1, 14-7, 15-1, 16-5, 17-2, 18-2, 19-2, 20-0, 
21-1, 22-1, 23-2. This comparison shows that the 
first class covers a wide breadth of information 
in terms of the number of factors involved. 

Similar data for the last order of “best” me- 
chanics (6 subjects) described in terms of 20 
items, shows the following numbers of items 


* Five of the factors had fewer than ten items 
listed because a lower limit for loadings of .100 
was applied. These five factors had 9, 9, 6, 7, and 
4 items respectively. Our comparison was re- 
stricted to the items listed. 
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TABLE 10 
20 ITEMS DESCRIPTIVE OF THE FouRTH ORDER oF “‘Best’’ MECHANICS (N =6) 


Item No. Response 


He can get along with people pretty well. 

He has a nice appearance. 

He knows how to use tools. 

When he works, he really works. 

He makes sure he does a good job. 

He’s just a “regular Joe.” 

If you leave him to doa job, you can always be sure he will get the job done. 
He’s just a normal guy. 

He is good at working on the plane. 

He seems to take pride in his work. 

He doesn’t have any sense of responsibility. 

He thinks he has had a raw deal in the service. 

He’s just careless. 

He achieves his aim in the wrong way. 

You have to stand over him and show him how to do everything. 
He shines in making decisions. 

He occasionally drinks as he works. 

No cooperation at all. 

He isn’t a credit to the Air Force. 

He “goofs-off”’ all day long. 


with relatively high loading on the factors, with 19-2, 20-0, 21-0, 22-2, 23-1. Analogous data for 
the factors listed first, followed by the number _ the third family of “poorest” mechanics (8 sub- 
of items: 1-3, 2-0, 3-1, 4-7, 5-1, 6-1, 7-1, 8-5, 9-2, jects described in terms of 28 items), is as fol- 
10-0, 11-0, 12-2, 13-1, 14-2, 15-1, 16-1, 17-1, 18-2, lows: 1-4, 2-4, 3-8, 4-0, 5-0, 6-6, 7-2, 8-0, g-5, 10-2, 


TABLE 11 
28 IreMs DESCRIPTIVE OF THE THIRD FAMILY OF “‘PooREst’’ MECHANICS (N=8) 


Item No. Response Item 


He works in a sloppy way. 

You wouldn’t feel safe unless you checked behind him. 

He is no “whiz’’ in his line. 

Most guys with that much experience know a lot more than he does. 
He isn’t a very careful worker. 

When he does a job, you know it will be done right. 

He is one of those ideal mechanics. 

Many times he works overtime even when he doesn’t have to do so. 
He knows what any modification of a Tech Order is about. 

He is a good man on any job. 

He deserves a promotion. 

You don’t have to worry about telling him what to do all the time. 
He continues on without taking many breaks. 

He will straighten a guy out and explain things to him. 

He seems just about the best I’ve seen as far as getting his work done. 
He will do an overtime job instead of leaving it to one of the others. 
He always has better ideas than most men. 

He makes sure he does a good job. 

He’s the energetic type who has to have something to do. 

His work is nearly 100% correct. 

He knows all the systems on an airplane. 

If he were an instructor, he could put it over pretty thoroughly. 

He knows his stuff. 

If you leave him to doa job, you can always be sure he will get the job done. 
He reads new Tech Orders at his first opportunity. 

He keeps working as long as there’s a man left standing. 

He can show you how to do the job right. 

His ambition will pay off. 


16 
Item 
I 
14 
17 
36 
68 
94 
97 
99 
107 
119 
30 
33 
35 
41 
53 
81 
89 
95 
104 
120 
Io 
12 
23 
27 
38 
II 
16 
19 
25 
34 
44 
47 
48 
49 
50 
65 
66 
68 
7° 
73 
79 
87 
97 
100 
103 
105 
106 
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11-0, 12-1, 13-0, 14-5, 15-6, 16-4, 17-1, 18-2, 19-0, 
20-0, 21-8, 22-1, 23-0. These comparisons between 
factors and types show that the descriptions of 
types cover a wide variety of information in 
terms of the factors involved. An additional ad- 
vantage is that information is given directly 
about specified subjects. It is mot necessary first 
to analyze factors and then assess each individual 
in terms of the factors, 

The gains realized in terms of an 
agreement analysis are not obtained with- 
out a sacrifice. The types cover such a 
breadth of information that it is difficult 
to do justice to the description of a type 
other than by listing the items which are 
descriptive of each type. We have already 
listed the items of a class (including the 
first two orders) to illustrate one type of 
“best” mechanics. Another type of “best” 
mechanic is illustrated by the last order 
of “best” mechanics isolated. The items 
of this order are listed in Table 10, with 


those answered “Yes” listed first and 
those answered following. 

Normally, an order would have 16 
individuals in a binary agreement analy- 
sis. The above order has only six because 
48 of the 54 subjects in the sample were 
classified into the first three orders. 

As an illustration of a type of “poor- 
est” mechanics, the items descriptive of 
the third family (8 subjects) of poorest 
mechanics isolated are given in Table 
11. 

The items descriptive of the one addi- 
tional order of “best” mechanics 
(N = 16), the one order of “poorest” me- 
chanics (N = 16), and the three remain- 
ing families of “poorest” mechanics 
(N = 8, 6, and 8, respectively) are listed 
in Tables 12-16. 


The descriptions of the fourth family (Table 


TABLE 12 
31 IreEMs DESCRIPTIVE OF THE THIRD ORDER OF MECHANICS (N= 16) 


Item No. Response 


Item 


He’s just careless. 


He wants to get in and do the job. 

When he does a job, you know it will be done right. 

He answers questions as best he can. 

He knows how to use tools. 

He always reports to work on time. 

You can always take his word for what he says he’s done. 

He deserves a promotion. 

You don’t have to worry about telling him what to do all the time. 
He sees decisions through to an end. 

He uses the right tools when he can get them. 

He makes sure he does a good job. 

He tries to find better ways of doing things. 

He stands up for what he believes in. 

If you leave him to doa job, you can always be sure he will get the job done. 
He usually remembers things which are explained to him. 

He can show you how to do the job right. 

His ambition will pay off. 

He is good at working on the plane. 

He likes being a mechanic. 

He gives good cooperation. 

He seems to take pride in his work. 

He works in a sloppy way. 

He is afraid to make a decision one way or another. 

He just stands around until you're done before offering to help. 


You have to stand over him and show him how to do everything. 

He doesn’t seem to care what happens to him when he gets out of service. 
No cooperation at all. 

He isn’t a credit to the Air Force. 

Sometimes you can trust him and sometimes you can’t. 

He “goofs-off”’ all day long. 


2 
II 
13 
17 
31 
39 
44 
47 
57 
60 

68 
72 
84 
97 
IOI 
105 
106 
107 
109 
118 
119 
10 
21 
28 
35 
53 
95 
104 
113 
120 
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14) appear to be, in general, complimentary of 
the mechanics even though the mechanics were 
selected as “poorest” ones. This apparent incon- 
sistency seems reasonable when we consider that 
the mechanics with whom we are concerned have 
been trained and selected for their jobs, and the 
terms “best” and “poorest” are relative and ap- 
plied in this setting to small groups of mechanics. 
Each supervisor, who served as an informant in 
describing a mechanic, was requested to select 
either a “best” or a “poorest” mechanic and de- 
scribe him. Each informant made his selection 
from those mechanics whom he had supervised 
in the last two years. Since the informants gen- 
erally supervised not more than six or seven men 
in any one assignment, the group from which 
each informant made his selections was small. 
And each group, as already pointed out, con- 
tained only men who had been trained and se- 


lected as mechanics. It is not surprising then to 
find that eight of the 46 men chosen as “poorest” 
mechanics are described rather complimentarily. 
They probably are reasonably good mechanics. 
This conclusion from the experimental sample 
is consistent with the results from our cross- 
validational studies (Tables 5, 6, and 8) where 
at least 11 per cent of the “poorest” mechanics 
invariably classified as “best” ones in terms of 
our pattern analytic method. 

However, there is a way in which the compli- 
mentary description of this family of “poorest” 
mechanics seems to differ from complimentary 
descriptions of “best” mechanics, The compli- 
mentary descriptions of “poorest” mechanics tend 
to avoid the most highly mechanic-centered de- 
scriptions, Instead, the compliments tend to be 
limited to the more general kinds of descriptions, 
such as: “He is a good man on any job”; “He 


TABLE 13 


38 Irems DESCRIPTIVE OF THE First ORDER (2 FAMILIES) OF ‘‘PoorEsT’’ MEecHANICS (N=16) 


Item No. 


Response 


<< 


Everybody likes him. 


a break. 


He knows his stuff. 


Y 
Y 
Y 
¥ 
N 
N 
N 
N 
N 
N 
N 
N 
N 
N 
N 
N 
N 
N 
N 
N 
N 
N 
N 
N 
N 
N 
N 
N 
N 
N 
N 


He accepts responsibility only when he must. 

He works in a sloppy way. 

His superiors should not give him a higher AFSC. 

He is no “‘whiz”’ in his line. 

Most guys with that much experience know a lot more than he does. 
He just stands around until you're done before offering to help. 

He doesn’t have any sense of responsibility. 

He isn’t a very careful worker. 

He does things without first finding out what will happen. 

When he does get over to work, he doesn’t do a thing. 

Instead of doing it, he says, “I'll think about it.”’ 

When he does a job, you know it will be done right. 

He is one of these ideal mechanics. 

He knows how to use tools. 

He knows what any modification of a Tech Order is about. 

He is a good man on any job. 

He deserves a promotion. 

You don’t have to worry about telling him what to do all the time. 
He continues on without taking many breaks. 

He will straighten a guy out and explain things to him. 

He seems just about the best I’ve seen as far as getting his work done. 
He sees his decisions through to an end. 


He will do an overtime job instead of leaving it to one of the others. 

He always has better ideas than most men. 

He makes sure he does a good job. 

He tries to find better ways of doing things. 

His work is nearly 100% correct. 

He knows all the systems on an airplane. 

There are times when he keeps working even though everyone else takes 


Everyone enjoys working with him. 


If you leave him to doa job, you can always be sure he will get the job done. 
He reads new Tech Orders at his first opportunity. 

He can show you how to do the job right. 

He is good at working on the plane. 

He gives good cooperation. 

He seems to take pride in his work. 
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TABLE 14 
22 IrEMS DESCRIPTIVE OF THE FouRTH FAMILY oF “PoorEst’’ MecHANics (N=8) 


Item No. Response Item 


If he were a crew chief, he would work right along with his men. 

He doesn’t just sit back with his hands on his hips. 

He is a good man on any job. 

You can take his word for what he says he’s done. 

You don’t have to worry about telling him what to do all the time. 
He seems just about the best I’ve seen as far as getting his work done. 
He uses the right tools when he can get them. 

He tries to find better ways of doing things. 

He stands up for what he believes in. 

What he doesn’t know he can find out for you one way or another. 
Everyone enjoys working with him. 

If you leave him to doa job, you can always be sure he will get the job done. 
He’s just a normal guy. 

He’s just careless. 

He has unpleasant characteristics. 

We have had a couple of “run-ins.” 

He doesn’t seem to care what happens to him when he gets out of service. 
He is just a kid. 

You have to explain every little step to him. 

He doesn’t care to mix with other people. 

He is pretty unhappy about being in maintenance, 

Instead of doing it, he says, “I'll think about it.” 


<< 


stands up for what he believes in”; “He is just 


a normal guy.” The “best” mechanics on the 
other hand are often described in terms of items 
which pertain more specifically to their particu- 
lar jobs as aircraft and engine mechanics, such 
as: “He knows how to use tools”; “He is good 
at working on the plane”; “He seems to take pride 
in his work”; “He likes being a mechanic.” 


Only six of the 43 items descriptive of 


the first class of “best’’ mechanics are 
included in the go items descriptive of 
the last order of “best” mechanics. In 
other words, we have evidence here of 
two quite distinct types of “best” me- 
chanics. 

Only 10 of the 22 items descriptive of 
the fourth family of “poorest” mechanics 


TABLE 15 


18 ITEMS DESCRIPTIVE OF THE FirTH FAMILy oF “Poorest”? MECHANICS (N=8) 


Item No. Response Item 


He doesn't know his job. 

His superiors should not give him a higher AFSC. 

He is no “whiz’’ in his line. 

Most guys with that much experience know a lot more than he does. 
He always reports to work on time. 

He does things without first finding out what will happen. 

He has just an average working knowledge of the job. 

He stands up for what he believes in. 

It takes him quite a while to learn. 

He is one of these ideal mechanics. 

He is a good man on any job. 

He deserves a promotion. 

He will straighten a guy out and explain things to him. 

He seems just about the best I’ve seen as far as getting his work done. 
He always has better ideas than most men. 

He knows all the systems on an airplane. 

He knows his stuff. 

He reads new Tech Orders at his first opportunity. 


<< 
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TABLE 16 
7 IreMs DESCRIPTIVE OF THE SIXTH FAMILY OF “PoorEst’’ MECHANICS (N=8) 


Item No. Response Item 


bf He accepts responsibility only when he must. 
He is no “whiz” in his line. 

He knows what any modification of a Tech Order is about. 
He is a good man on any job. 

He deserves a promotion. 

He always has better ideas than most men. 

Instead of doing it, he says, “‘I’ll think about it.” 


are included in the 43 items descriptive 
of the first class of “best’’ mechanics, and 
only two of the 22 items for the poorest 
family are included in the go items 
descriptive of the last order of “best” me- 
chanics; the numbers of items which are 
not common suggests that “best” and 
“poorest” mechanics differ in ways other 
than merely being opposite. 


Interassociations Between Factors and 
Types 

In the above section on interpreting 
the types, comparisons were made with 
results from an unrotated, square-foot 
factor analysis of common data. In the 
present section, we wish to review the 
comparison for another purpose. We wish 
to call attention to interassociations be- 
tween types and factors. The present 
evidence as to the exact nature of the 
associations is not maintained to be con- 
clusive, and the data available are not 
the most crucial kind. The responses are 
descriptions by supervisors of mechanics, 
rather than known characteristics of me- 
chanics. Consequently, whether the in- 
terrelationships derived from them reflect 
only the opinions of informants or are, in 
fact, characteristic of mechanics, is not 
known with certainty at this time. The 
factor analysis was unrotated, and by a 
square-root method rather than by a 
more exacting procedure. Likewise, the 
agreement analysis was by the binary 


method rather than the more exacting 
and complete procedure (6, 7). 


Of the three types of “best” mechanics re- 
ported, the large class of them (32 subjects and 
43 items) is heavily involved in the first factor in 
that all 10 of the items with the highest loadings 
on this factor are contained in the 43 items 
descriptive of this type. Each of the next two 
factors (in order of amount of variance accounted 
for) have none of their ten highest-loaded items 
in the 43 items descriptive of the large class of 
32 subjects. This class had a disproportionate 
amount of its variance accounted for in the first 
factor, and consequently other types were in- 
volved in the second and third factors. In the 
fourth factor, the large class reappears together 
with the other two types of “best” mechanics. 
Eight of the ten highest-loaded items on this fac- 
tor are included in the 43 items descriptive of 
the large class, and five of these plus two others 
of the ten highest-loaded items are among the 
20 descriptive of the last order of “best” me- 
chanics. The relative association of these types 
with other factors can be seen in the figures re- 
ported earlier which show the number of high- 
loaded items for each factor from each type. 
Similar results were reported for the third family 
of “poorest” mechanics (8 subjects and 28 items). 
They reveal that this type is heavily involved in 
factors 3 and 21, with 8 high-loaded items from 
each, The two “best” types reported had only 
one item with a high loading in either of these 
two factors. The high degree of association be- 
tween factor 21 and this “poorest” type is even 
more striking when we consider that this factor 
accounts for only 0.8 per cent of the variance in 
the data analyzed, and the agreement analysis of 
“poorest” mechanics was done on only 46 of the 
428 subjects on which the factor analysis was 
performed. 

In this comparison of types with factors we 
have illustrated our points with one class of 32 
subjects and one order of six subjects from “best” 
mechanics, making a total of 38 “best” mechanics 
and leaving one order of 16 “best” mechanics 
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unaccounted for. Analogously, we have used only 
one family of “poorest” mechanics, leaving four 
families of eight mechanics and one of six me- 
chanics unaccounted for. Data analogous to that 
already reported are summarized for all of the 
above-mentioned types in Table 17. This table 
reports the 23 factors which were isolated in the 
unrotated square root solution, listed in order 
of amount of variance accounted for. All of 
these factors except the 10th, 14th, goth, 22nd, 
and 2grd had 10 items with loadings above 0.100. 
These specified factors had only 9, 9, 6, 7, and 4 
items respectively with loadings above 0.100. 
Table 17 is concerned with the 10 highest-loaded 
items for each factor except those which had 
fewer than 10 above loading of 0.100. For these, 
it is concerned only with those above 0.100. The 
table also lists types of “best” and “poorest” me- 
chanics derived from the agreement analysis. 
These are the particular ones used in the re- 
liability study, reported earlier. For each type, 
the *..ble shows the number of high-loaded items 
on each factor. The table reveals, as already 
illustrated, that types are associated with factors. 

Table 17 contains an additional lesson. It 
shows a way in which factor and agreement 
analyses can be used jointly in interpreting be- 
havior. For this purpose, four or more highly- 


21 


loaded items common to a factor and a type are 
taken to mean that a type is relatively high or 
low on a factor. For example, the four items 
listed under Factor 5 and opposite Type 1 mean 
that this “best” type is relatively high on this 
factor. Analogously, the four items under Factor 
8 and opposite Type 4 mean that this “poorest” 
type is relatively low on this factor. The stand- 
ings are all in the direction which would be 
expected from the names of the factors and the 
labels of “best” and “poorest” which are applied 
to the types, except for Type 6, the fourth 
family of “poorest” mechanics, which is com- 
plimentarily described as we have already 
pointed out. 

Applying the procedure just outlined, we see 
that Type 1 subjects are responsible; they are 
not lazy or weak in character. They do not fail 
to use knowledge effectively, are industrious, re- 
member well, are socially acceptable, are practi- 
cal workmen and do not lack craftsmanship. Type 
4 subjects, on the other hand, are low on re- 
sponsibility and willingness to work; they are 
lazy and fail to use knowledge effectively; they 
are not industrious and do not remember well; 
they are not practical workmen or craftsmen, 
and they lack both intellectual capacity and 
job knowledge. This “poorest” type is defined 


TABLE 17 


How Items DescriPTIvVE OF EACH TypE ARE DISTRIBUTED AMONG THE 
Factors IN TERMS OF THE HIGH LOADINGS*® 


Factor Titles and Numbers 


2. Interest in Aircraft Maintenance 


1. Sense of Responsibility 
3. Willingness for Work 


5. Weakness of Character 


6. Failure to use Knowledge Effectively 


7. Teaching Capacity 


4. Laziness and Lack of Initiative 
8. Industriousness 


No. Persons in Type 


10. Self-Control 

11. Inexperience 

12. Social Acceptability 
13. Lack of Morale 

14. Practical Workmanship 
15. Intellectual Capacity 
16. Lack of Craftsmanship 
17. Personal Pleasantness 
18. Tendency to Mediocrity 
19. Antisociability 

20. Lack of Self-Control 
21. Job Knowledge 

22. Unlabeled 


“Best” 
1—1st Class 
2—3rd Order 
3—4th Order 


adn 
woo 


oro 
oon 


“Poorest” 
4—1st Order 
5—3rd Family 
6—4th Family 
7—sth Family 
8—6th Family 


® For example, of the 43 items which describe the 1st class of “best” mechanics, ro of them are the same items which have the 
highest ten loadings on factor 1; none of the 43 is re the first ro in terms of loadings on factor 2; the same is true for factor 3; 


eight of the 43 are among the first 10 for factor 4, etc 


ll of the factors except the roth, r4th, 20th, 22nd, and 23rd had ro items 


with loadings above 0.100. These latter factors had only 9, 9, 6, 7 and 4 items respectively with loadings above 0.100. In the case of 
these factors, the comparisons were limited to the numbers of items just specified. In all other factors, the ten items with the ro 


highest loadings were used. 
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as opposite to the above “best” type in all 
but two factors (weakness of character and social 
acceptability) of the eight used to define the 
“best” one. In addition, the “poorest” type is 
defined as deficient in three factors which were 
not used in defining the “best” type (willingness 
to work, intellectual capacity, job-knowledge). 

In an analogous fashion, the other types can 
be described concisely from Table 17. In de- 
scribing these types, it should be remembered 
that the raw data are descriptions given by 
supervisors; the patterns of descriptions may 
exist only in the minds of the supervisors or 
both in their minds and in the behavior of 
mechanics. Consequently, the patterns are not 
known at this time to be characteristic of the 
behavior of mechanics. They are here reported 
only as characteristic of descriptions of mechanics 
by supervisors. Additional research is required to 
determine whether they would also be obtained 
from objective assessment of behavior. 

We have shown that some items are internally 
consistent in both factor and pattern analytic 
analyses, However, there were nine items which 
were internally consistent for some of the types 
by agreement analysis but did not appear with 
substantial factor loadings on any of the factors. 
These items are as follows: 


6 If he were a crew chief, he would work 
right along with his men. 

13 He answers questions as best he can. 

15 He will climb into one of the airplanes 
and sleep. 

34 He is a good man on any job. 

55 If there is something he likes to do, 
he does it faster than anyone else would. 

57 He sees decisions through to an end. 

67 He is very glad to help someone else. 

73 His work is nearly 100% correct. 

75 He doesn’t seem to care what happens 
to him when he gets out of service. 


The one thing that these items appear to 
have in common is that they are extreme state- 
ments. It seems that some items can be highly 
dependable for describing some one or more 
categories of subjects even though they are not 
generally dependable for describing all subjects 
in terms of any one factor. 

We searched for items which had relatively 
high loadings but were not included in any of 
the types on which the reliability study was done. 
(The search was made at an intermediate level 
of classification.) Relatively high loadings were 
defined as sufficiently high so that an item was 
included in interpretation of a factor (either in 
the first ten items in terms of loadings for a 
factor or a loading above .100 in the case of those 
factors which did not have 10 items with loadings 
above this value). There were go items with 
relatively high loadings (as just defined) which 


did not appear in any one of the types on 
which the reliability study was done. In addition, 
there were 28 items with relatively high loadings 
which were not included in descriptions of 
“best” types and 25 other items with relatively 
high loadings which did not appear in descrip- 
tions of “poorest” types. These three categories 
of items plus the nine items already reported 
(which showed maximum relevancies for the 
types even though they had no relatively high 
loadings) make a total of g2 items on which there 
was considerable difference in the assessments 
of internal consistency by the two methods 
(agreement analysis and factor analysis). 

Even though we grant that relative size of 
factor loadings on unrotated square-root factors, 
treated singly, is not a good estimate of internal 
consistency, the large discrepancies here reported 
show that there is a difference in the internal 
consistency of items as measured across all sub- 
jects (factor analysis) versus its assessment within 
selected’ categories of subjects (agreement 
analyis). In addition, these results show that 
many items differ in their validities for assessing 
factors and types. In short, these findings empha- 
size a new way in which reliability and validity 
are relative concepts; they are relative to their 
assessment across all subjects versus their assess- 
ment within selected categories of subjects. 

The above results reveal the need for a more 
crucial comparison of results from factor and 
agreement analysis of common data, using the 
most refined technique of each method, together 
with data especially selected for the purpose of 
the comparison. 


Ill. A Cririgue oF THE METHOD 


It will be helpful now to react critically to the 
method just outlined and applied. The method 
is made computationally feasible by an arbitrary 
restriction; there is no reason (other than 
simplicity of classification) why the types at any 
one level should be composed of only two 
patterns from the next lower level. A more 
complicated version of agreement analysis avoids 
this arbitrary restriction (7). However, this latter 
method is laborious and it is felt that the present 
binary method is probably sufficiently exacting 
for our present state of knowledge. 

Another problem of the method is that it sets 
no level of statistical significance which an agree- 
ment score must meet before a classification can 
be based on the score. The method circumscribes 
this problem by the following logic: It assumes 
types, and argues that if they exist for all 
patterns of response, then every pattern of re- 
sponses must be classified and a solution is to 
give each pattern its best classification. This is a 
much less complicated solution than would be 
involved if the level of significance were required 
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to be computed for every classification of every 
pattern. 

One could wonder whether or not “yes-no” 
answers are sufficiently dependable as a basis for 
classification. A reasonable reply to this kind of 
inquiry is to state that we do not have sufficient 
information of an appropriate kind on which to 
base an answer. Studies on the reliability of 
“yes-no” answers have been based on their de- 
pendability as assessed across all subjects of a 
sample. Agreement analysis recognizes the possi- 
bility of differential reliability of an answer; 
an answer may be highly reliable and dependa- 
ble for some categories of subjects and not others. 
Agreement analysis enables us to determine 
whether or not “yes-no” answers have sufficient 
reliability of this kind to continue to justify the 
application of the method to them. Even if 
“yes-no” answers were found wanting, the 
method might still prove helpful if applied to 
categorized data, as obtained by dividing scores 
on several continuous variables into class 
intervals. 

Agreement analysis does not allow us to assign 
every type a point in space in relation to every 
other type, and thus to give it a number which 
would reveal its location. Whether or not one 
wishes to do this depends on one’s theory of the 
nature of types. If one wishes to assume that 
the same response in two different patterns has 
the same meaning, then it is reasonable to use 
the overlapping responses of patterns to assign 
types locations in space; this can be done in a 
method called linkage analysis (8). However, if 
one wishes to assume that the meaning of a re- 
sponse depends to a considerable extent on the 
pattern in which it occurs, then there would 
appear to be no basis for allocating the types to 
spatial locations on the basis of overlapping re- 
sponses in their patterns. 

Agreement analysis does not settle the differ- 
ences in theoretical issues about types. It is, 
however, a method which should prove of as- 
sistance along with other metheds in investigat- 
ing some of the theories about types, and even 
in studying whether or not it seems worth while 
to assume types. 


IV. SUMMARY 

This paper has applied an abbreviated 
version of a method of analysis, called 
agreement analysis, for the isolation and 
classification of types into a hierarchical 
system. An unusual feature of the method 
is that the terminal table on which in- 
terpretations are based contains every 
bit of information represented in the 
raw data; information is abstracted and 
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generalized at successive levels, and the 
level most pertinent for a given purpose 
can be selected for further study. In 
addition, the method of analysis can 
handle all kinds of interrelations in data 
(linear, nonlinear, and disjunctive), and 
it also provides for giving various inter- 
pretations to any particular answer to an 
item, depending on the combination or 
pattern of other answers with which it 
occurs. 

Illustrative applications of agreement 
analysis to descriptions of mechanics 
selected as “best” or “poorest” confirm 
the following hypotheses: The method 
yields both reasonably dependable and 
valid results. Descriptions of “best’’ me- 
chanics can be differentiated from those 
of “poorest” ones on the basis of type 
membership. In addition, the results sug- 
gest the hypothesis that categories of 
people, such as “best” and “poorest” me- 
chanics, can be most clearly differentiated 
in terms of many types, each type con- 
taining only a few highly similar sub- 
jects, rather than a few types containing 
many subjects. In terms of our data, 
there appear to be many types of “best” 
mechanics and many types of “poorest” 
mechanics—but no all-inclusive single 
type of “best” mechanic which differs 
from an all-inclusive type of “poorest” 
mechanic. Hypotheses of this kind can 
be further investigated by the method of 
agreement analysis which has been out- 
lined here. 

Results from our agreement analyses 
were compared with results from a factor 
analysis of common data. The compari- 
son illustrated that the two methods 
classify data differently but not entirely 
independently; the factors and types 
which resulted are interassociated. Both 
methods yield results which appear to 
be meaningful and they can be used 
jointly in the interpretation of behavior. 
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However, some test items can have in- 
ternally consistent responses for some 
categories of subjects in terms of an 
agreement analysis even though they are 
not sufficiently consistent across all sub- 
jects to yield any factor loadings of any 
consequence. These items appear to have 
value for selecting out rare subjects, such 
as a mechanic who is so expert that he is 
always 100 per cent correct; or, on the 
other hand, a poor mechanic who is so 
worthless that he would climb into an 
airplane and sleep. 

Conversely, other items can be so in- 


ternally consistent across all subjects that 
they have relatively high loadings and 
still not be retained as descriptive of 
any one of the predominant types at 
intermediate levels of classification. Con- 
sequently, the internal consistency of 
many items in terms of predominant 
types is quite different from their in- 
ternal consistency in terms of factors. 
These issues on reliability of items under 
the two methods of analysis require fur- 
ther study by the most refined techniques 
of factor and agreement analyses with 
items especially chosen for the purpose. 
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